MIRACLE at ImageCLEFphoto 2007: Evaluation of Merging Strategies for Multilingual and Multimedia Information Retrieval

This paper describes the participation of MIRACLE research consortium at the ImageCLEF Photographic Retrieval task of ImageCLEF 2007. For this campaign, the main purpose of our experiments was to thoroughly study different merging strategies, i.e. methods of combination of textual and visual retrieval techniques. While we have applied all the well known techniques which we had already used in previous campaigns, for both textual and visual components of the system, our research has primarily focused on the idea of performing all possible combinations of those techniques in order to evaluate which ones may offer the best results and analyze if the combined results may improve (in terms of MAP) the individual ones. The system includes three main modules. On one hand, apart from the search engine (Xapian or Lucene), the textual retrieval module includes parsers, stemming, stopword filtering, proper noun detection and semantic expansion components. On the other hand, the visual retrieval module is based on two well-known content-based engines: GIFT and FIRE. Finally, the merging module allows to use different operators (AND, OR, LEFT, RIGHT) to combine the outputs of the two previous subsystems and to calculate the result relevance based on different metrics (max, min, avg, max-min). We finally submitted 110 multilingual textual (text-based) runs, 22 visual (content-based) runs and 21 mixed runs. Results in general show a poor performance for all groups, due to the characteristics of the image collection and the difficulty of the defined topics. The most interesting conclusion is that the defined merging strategies are successful as our best mixed experiment outperforms both the textual and visual experiments in which it is based, using the LEFT operator for the combination along with the max-min metric for computing the relevance.